-
Notifications
You must be signed in to change notification settings - Fork 350
dax: set instance lifespan from prepare to reset #10503
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
At present, dax instance is initialized and allocated inner buffer on module_init(), and released on module_free(). The memory requirement for the inner buffer per instance is quite large. The existing implementation has 2 pipelines containing dax. Even though the host is restricted from processing stream on both pipelines simultaneously, they may coexist with each other under some circumstances e.g. the interim when switching PCM device from one to the other, which may drain out the memory. This commit changes the timing of instance allocation/deallocation to module_prepare()/reset() respectively. That is, dax instance only occupies the inner buffer memory when processing. Signed-off-by: Johny Lin <[email protected]>
|
Will need review by Dolby folks @checkupup |
lgirdwood
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@johnylin76 IIUC, you have 2 pipelines today, each with a DAX instance that allocates a large inner buffer. I'm not following why this fixes your issue, I can only assume that this change only allocates 1 mutually exclusive inner buffer as perhaps allocating > 1 inner buffer was failing ?
| @@ -476,6 +476,51 @@ static int sof_dax_free(struct processing_module *mod) | |||
| dax_buffer_release(mod, &dax_ctx->tuning_file_buffer); | |||
| dax_buffer_release(mod, &dax_ctx->input_buffer); | |||
| dax_buffer_release(mod, &dax_ctx->output_buffer); | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since only persist and scratch buffers are allocated during entablishment phase, we could be only responsible for releasing them during the destruction phase:
static void destroy_instance(struct processing_module *mod)
{
struct sof_dax *dax_ctx = module_get_private_data(mod);
dax_free(dax_ctx);
dax_buffer_release(mod, &dax_ctx->persist_buffer);
dax_buffer_release(mod, &dax_ctx->scratch_buffer);
}| { | ||
| struct sof_dax *dax_ctx = module_get_private_data(mod); | ||
|
|
||
| destroy_instance(mod); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if sof_dax_reset can always be called before sof_dax_free phase, destruction may be not necessary here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Theoretically yes, but I don't feel we should rely on that assumption.
The memory free() function of C causes no harm if the input pointer is null. Your dax_buffer_release() implementation is handled well so they are good to be called more than once.
My only concern is that I am not sure if calling dax_free() twice is also good because this is a private call
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if so, following releases are also required here:
dax_buffer_release(mod, &dax_ctx->tuning_file_buffer);
dax_buffer_release(mod, &dax_ctx->input_buffer);
dax_buffer_release(mod, &dax_ctx->output_buffer);dax_free() is safe for multiple frees.
| ret = establish_instance(mod); | ||
| if (ret != 0) | ||
| return ret; | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be better to place establish_instance after check_media_format to avoid unnecessary destruction if check_media_format fails.
On the other side, we need to add the option CONFIG_IDC_TIMEOUT_US=50000 to the file app/overlays/[ptl/mtl]/dax_overlay.conf to avoid IPC_TIMEOUT errors, because dax_init takes more than 20 ms, exceeding default IDC_TIMEOUT value (10ms).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's also I worried about. But I don't get why dax_init() takes so long because at that moment DAX didn't even get the configuration data (from dax_param.bin). Or does DAX make the advance calculation on dax_init() such as configuration prediction?
We have verified without changing CONFIG_IDC_TIMEOUT_US and didn't get timeout problem (I assume the timeout should lead to a stable failure if it really cares). Could you check with that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can set DOLBY_DAX_CORE_ID as 1 (or other different cores with pipeline) to trigger IDC timer.
dax_init will perform a large number of initialization operations based on the provided memory and load some static data.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. You are correct "IDC" is for the inter-CPU communication.
As for dax_init(), do you mean "loading static data" to memory allocated on runtime for speeding the later computation? I remember we didn't use DRAM for ADSP in the current device so it still sounds weird to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
my bad for clarifying this.This refers to loading static data from RAM (decalared as static arrays and stored in .data section) into runtime buffer and performing initializations for features in it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@johnylin76 have you looked at putting all instance shared objects in BSS, DATA and RODATA sections and managing in the module wrapper. e.g.
struct instance_shared_data {
int this;
void *that;
};
/* shared instance data for all dax modules*/
static struct instance_shared_data *data; /* will be in module BSS that will be shared with all instances */
static atomic_t instance_count = 0;
int dax_init()
{
/* check number of instances and create shared data if needed */
if (atomic_add(instance_count, 1) == 1) {
/* create the data */
}
/* rest of init */
}
void dax_destroy()
{
/* check instance count and free shared data if last user */
}The IPC logic enforces locking and serialization so a reference count would allow managing any shared data between dax instances since they are mutually exclusive.
I also assume both dax instances will run on same core ?
| struct comp_dev *dev = mod->dev; | ||
|
|
||
| /* dax instance will be established on prepare(), and destroyed on reset() */ | ||
| destroy_instance(mod); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Simultaneously release following buffers here:
dax_buffer_release(mod, &dax_ctx->tuning_file_buffer);
dax_buffer_release(mod, &dax_ctx->input_buffer);
dax_buffer_release(mod, &dax_ctx->output_buffer);Becasue these buffers are also allocated in sof_dax_prepare. If a lifecycle of dax instance is from prepare to reset phrase, then I believe that all resources created in sof_dax_prepare should have the same lifecycle.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. The better practice is releasing all runtime buffers when reset.
checkupup
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @johnylin76 , I added some comments
Yes. The majority comes from the inner (scratch and persist) buffer. The memory will be drained out when both DAX instances allocate that. By host control, we can make sure 2 DAX instances won't run process simultaneously, although they may be lived together. Consequently, we let DAX allocate the inner buffer when triggered start to process (and deallocate when stopped) |
|
Hi @checkupup , I just found that there is one critical thing to be cleared: How does the mixer control configuration persist through each lifespan of the dax instance? AFAIK, the current dax impl uses For the CURRENT dax impl, the lifespan of an instance is aligned with module_adapter, hence the config is always synced. But for NEW dax impl, instances are started over by new request while the configuration is unchanged (no IN SHORT, the module instance should sync the complete configuration during the establishment. Accordingly, the wrapper should store the complete and up-to-date configuration for syncing. (another design idea is that you can simulate a host that sets the configuration once again when establishing a new instance). Please help to take over this PR to add the fix for the configuration persistence. Thanks |
| @@ -476,6 +476,51 @@ static int sof_dax_free(struct processing_module *mod) | |||
| dax_buffer_release(mod, &dax_ctx->tuning_file_buffer); | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tuning_file_buffer should not be released by destroy_instance (or implements a way to recover the buffer data on establishment). Move to sof_dax_free
I think if we can use BSS, DATA and RODATA object with some reference counting we can have persistance. Today the kernel will still export 2 sets of kcontrols (one for each instance) as it reads the topology and has no way to differentiate the dax module instances. However, a topology change to have a "dax no controls" module that does not export any kcontrols would work (since the current dax instance would create and share these with other dax instance). |
commit message
At present, dax instance is initialized and allocated inner buffer on module_init(), and released on module_free(). The memory requirement for the inner buffer per instance is quite large.
The existing implementation has 2 pipelines containing dax. Even though the host is restricted from processing stream on both pipelines simultaneously, they may coexist with each other under some circumstances e.g. the interim when switching PCM device from one to the other, which may drain out the memory.
This commit changes the timing of instance allocation/deallocation to module_prepare()/reset() respectively. That is, dax instance only occupies the inner buffer memory when processing.
This commit is to fix the following bug on the targeting Google device:
bug report
Steps to Reproduce:
Expected behavior:
Switch to the internal speaker output with sound,when connecting headphones to play music.
What do you see instead?
Switch to the internal speaker output without sound,when connecting headphones to play music.
Fail rate:
2 out of 2 units,6 out of 40 Times